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PREFACE 


This  report  covers  the  design  and  development  of  a  conceptual 
prototype  for  a  computer-based  training  system  to  be  used  for 
command  and  control  decision-making  training  research. 

The  project  was  guided  by  Dr.  LorRaine  T.  Duffy  and  executed  by 
the  Logicon  Tactical  and  Training  Systems  Division  in  San  Diego, 
California,  under  the  direction  of  Mr.  Greg  Butler.  Logicon  team 
members  included  Ms.  Jane  McGarvey  and  Ms.  Gail  Slemon.  Particular 
thanks  go  to  the  Air  Force  personnel  of  PACAF/DOC,  including  Col.  Robert 
Hammond,  Lt.  Col.  Donald  Motz,  Major  Keith  Erickson,  Major  Wayne 
Sterzinger,  and  Mr.  Gary  Ward. 


Apple,  Macintosh,  and  HyperCard  are  registered  trademarks  of 
Apple  Computer,  Inc. 

SuperCard,  SuperEdit,  Super  3D,  and  SuperPaint  are  registered 
trademarks  of  Silicon  Beach  Software,  Inc. 

Windows  3.0  is  the  registered  trademark  of  Microsoft 
Corporation. 

Capture  is  the  registered  trademark  of  Mainstay,  Inc. 

Toolbook  is  the  registered  trademark  of  Asymetrix  Corporation. 

IBM  is  the  registered  trademark  of  International  Business 
Machines  Corporation. 
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I .  SUMMARY 


The  decision-making  skills  which  are  key  to  most  management  jobs  are 
generally  acquired  through  "experience”  rather  than  formal  training. 
While  procedures,  techniques,  and  formulas  can  be  explained  and 
demonstrated  to  a  management  trainee,  it  is  unlikely  that  he  or  she  will 
be  able  to  reliably  and  effectively  apply  them  on  the  job  without 
practical  experience.  Training  systems  attempt  to  provide  that 
experience  prior  to  the  job  in  a  realistically  controlled,  efficient, 
and  economical  environment. 

This  management  training  problem  is  prevalent  in  the  command  and 
control  community.  This  project  addresses  that  problem  with  an 
experimental  training  system  intended  to  provide  "experience"  to 
those  officers  at  Headquarters  (HQ)  Pacific  Air  Forces  (PACAF)  who  are 
preparing  to  perform  the  job  of  Executive  Officer  in  the  Crisis 
Action  System  (CAS).  The  targeted  personnel  are  typically  majors 
or  lieutenant  colonels  with  operational  experience.  Their  job  will 
be  to  filter  all  incoming  messages  at  the  command  and  control 
center,  determine  the  significance  of  each,  and  assign  and  track 
responding  actions  within  the  CAS  staff.  This  includes  drafting  new 
messages  and/or  coordinating  communications  in  person  or  by  phone 
with  a  wide  variety  of  people  and  agencies,  both  within  and  outside  of 
the  command  center.  Because  the  tasks  are  necessarily  "intellectual" 
and  not  "manual,"  it  is  difficult  to  train  without  reference  to  the 
milieu  in  which  the  tasks  are  conducted. 

A  common  method  of  providing  training  for  this  CAS  Executive  job 
(as  well  as  other  jobs  in  CAS)  is  to  conduct  exercises  which 
simulate  in  vivo  involvement  in  and  response  to  crises.  While  these 
live  exercises  may  be  the  most  effective  method  of  training,  they 
are  infrequent,  expensive,  and  can  only  serve  a  limited  number  of 
trainees . 

The  product  of  this  project,  PACEXTRA  (PACAF  CAS  EXec  TRAining) 
is  an  experimental  system  designed  to  provide  individual  CAS 
Executive  trainees  with  private  exercises  on  intellectual  and 
coordinative  tasks,  conducted  interactively  on  a  microcomputer.  It 
presents  them  with  incoming  messages,  phone  calls,  and  visits  from 
CAS  staff  members,  and  permits  them  to  respond  with  messages, 
phone  calls,  and  tasking  of  staff  members.  The  system  also  provides 
trainees  with  reference  materials  and  forms  on  which  to  log  events  and 
track  suspenses.  Trainees  are  provided  varied  levels  of  assistance 
and  prompts,  including  advice  and  feedback  on  actions  to  be  taken  as 
a  result  of  specific  exercise  events. 

One  of  the  more  significant  features  of  PACEXTRA  is  a  built-in 
authoring  system  which  makes  it  possible  for  any  CAS  Exec  Subject 
Matter  Expert  (SME),  responsible  for  training  other  CAS  Exec 
trainers,  to  create  and  revise  training  exercises  as  needed.  This  is 
done  through  a  graphic,  user-friendly,  interactive,  on-screen  process 
which  requires  no  special  skills  and  no  programming  background. 
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Although  PACEXTRA  trains  only  PACAF  CAS  Exec  officers,  it 
exemplifies  a  means  of  economically  and  efficiently  providing 
exercise-based  individualized  training  for  almost  any  event-driven 
job.  This  instructional  approach  and  medium  could  serve  any 
technical  training  program  from  store  clerk  (whose  tasks  tend  to  be 
manual  and  procedural)  to  space  vehicle  crew  member  (whose  tasks 
are  more  intellectual,  coordinative,  and  unmeasurable). 
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II.  PROBLEM 


The  process  of  making  good  decisions,  on  any  job,  is  a  complex 
mixture  of  setting  goals,  collecting  information,  defining 
alternatives,  weighing  options,  and  choosing  solutions.  While  rules, 
guidelines,  and  procedures  can  be  established  to  aid  in  this  process 
for  a  particular  job,  and  training  for  the  job  can  explain  and  even 
demonstrate  proper  application  of  these,  trainees  seldom  reach 
uniform  proficiency  in  decision-making.  Decision-making 
proficiency  is  a  function  of  knowledge,  skills,  and  rules  acquired 
through  varying  operational  experiences.  In  this  case,  an  attempt 
was  made  to  focus  on  the  knowledge  and  decision-making  acquired 
through  in  vivo  experimental  training,  as  opposed  to  rule-  or 
skill-based  decision-making. 

The  PACAF  CAS  is  formed  only  during  times  of  crisis  or  exercise.  The 
job  of  the  CAS  Executive  Officer  is  to  assist  the  CAS  Director  in 
managing  the  CAS  and  its  staff  of  up  to  40  or  50  people.  The  primary 
CAS  Exec  duties  are  to  screen  all  incoming  correspondence  (a  majority  of 
which  are  messages)  to  determine  what  action,  if  any,  is  required  of  the 
CAS;  assign  that  action  to  one  or  more  staff  members;  and  track  the 
progress  of  that  action  to  its  completion.  Secondary  duties  include 
compiling  reports  and  briefings  for  Headquarters  or  other  agencies. 

The  job  of  PACAF  CAS  Exec  is  filled  by  officers  assigned  to  the 
Directorate  of  Command  and  Control  (DOC)  under  the  Deputy  Chief  of 
Staff  for  Operations  (DO).  Several  people  must  be  available  for 
multiple  12-hour  CAS  shifts.  These  shifts  are  combined 
with  other  normal  duty  commitments,  and  are  subject  to  typical 
reassignment  rotations.  This  creates  the  need  for  ongoing  training 
of  new  CAS  Execs  and,  in  turn,  demands  on-the-job  experience 
which  previously  would  only  be  gained  during  actual  crises  or  large- 
scale  exercises. 

III.  APPROACH 

Overview 

The  approach  taken  to  experimentally  address  the  problem  of 
providing  CAS  Exec  trainees  with  realistic  job-like  experience  has 
been  to  employ  state-of-the-art  off-the-shelf  microcomputer 
software  and  hardware  to  develop  individualized  job-like  exercises. 

Apple  Macintosh  computers  were  chosen  as  the  training  environment 
due  to  their  availability  in  the  command  center.  The  exercises 
place  CAS  Exec  trainees  in  a  full-color  graphic  environment  which 
replicates  the  PACAF  CAS.  A  scenario  of  realistic  crisis-related 
events  is  presented  (see  Figure  1).  The  trainee  is  expected  to  judge 
the  significance  of  each  event,  determine  if  CAS  Exec  action  is 
appropriate,  and,  if  so,  take  action  using  on-screen  tools.  These 
tools  replicate  real-life  means  of  decision  execution.  They  include 
the  means  for  making  phone  calls,  sending  messages,  tasking  staff 
members,  taking  notes,  logging  events,  tracking  suspenses,  using 
references,  and  compiling  situation  reports  (SITREPs)  (see  Figure  2). 

The  trainee  can  also  obtain  advice  and  feedback  on  the  appropriate 
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actions  for  each  event  (see  Figure  3).  This  is  available  at  various 
expertise  levels,  depending  on  trainee  experience  and  training  need. 

The  training  system,  PACAF  CAS  EXec  TRAining  (PACEXTRA),  also 
incorporates  an  interactive  authoring  system  which  allows  SMEs  to  create 
and  refine  training  exercises.  This  system  facilitates  the  creation  of 
individualized  exercises,  composed  of  a  chain  of  custom  events,  through 
a  graphic  interface  which  requires  no  programming  and  minimum  training. 


Figure  1.  PACEXTRA  puts  CAS  Exec  trainees  in  a  graphic 
environment  representative  of  the  actual  PACAF  CAS.  (Note:  The  quality 
and  content  of  the  actual  screen  display  were  degraded  significantly  in 
de-colorizing  it  for  this  printing.) 
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Figure  2.  PACEXTRA  Introduction  Screen 

Figure  2.  Trainees  interact  with  simulated  CAS  staff  members 
through  graphic  screens  which  incorporate  two- 
way  textual  conversations.  (Not-ej  The  quality  and 
content  of  the  actual  screen  display  were  degraded 
significantly  in  de-colorizing  it  for  this  printing.) 
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Figure  3.  A  trainees  review  advice  and  feedback  through  a  display 
listing  the  type,  recipients,  and  content  of  their  anticipated 
communication. 

Planning 

Planning  for  PACEXTRA  involved  three  major  steps: 

1.  Confirming  the  CAS  Executive  Officer's  job  as  the  best 
subject  matter  for  the  experimental  training  system, 

2.  Determining  the  instructional  method  to  be  used,  and 

3.  Selecting  the  appropriate  hardware  and  software. 

Subject  Matter  Confirmation 

This  project  is  the  second  in  a  two-phase  program  to  experimentally 
enhance  CAS  decision-making  training  at  HQ  PACAF.  The  first  phase  was 
the  PACAF  Command  Control  Information  and  Training  System  (PACCITS). 
PACCITS  addresses  the  training  and  information  retrieval  needs  of  the 
entire  PACAF  CAS  by  providing  a  random  access  information  retrieval 
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system.  This  black  and  white  system  employs  the  HyperCard  software 
which  is  provided  on  all  Macintosh  computers  in  the  CAS. 

During  the  development  of  PACCITS,  the  Air  Force  Human  Resource 
Laboratory,  Ground  Operations  Branch  staff,  PACAF/DOC  staff,  and  Logicon 
representatives  considered  a  means  of  applying  more  advanced  forms  of 
hypermedia  to  focus  training  on  a  specific  CAS  position  which  is  heavily 
decision-event  driven.  Primary  factors  considered  were  the  criticality, 
complexity,  formal  definition,  and  turnover  of  various  CAS  staff 
positions.  It  was  decided  that  the  CAS  Exec  position  was  the  best 
candidate  because:  1)  it  manages  the  rest  of  the  staff,  2)  it  often  is 
involved  in  all  CAS  activities,  3)  it  follows  published  procedures,  4) 
it  is  mainly  dependent  upon  the  decision-making  skills  of  the  officer, 
and  5)  it  is  always  fully  manned  (by  a  large  pool  of  qualified 
personnel)  whenever  the  CAS  is  active. 

Instructional  Method  Determination 

The  CAS  Exec's  job  is  primarily  one  of  reviewing  each  piece  of 
incoming  information  (mainly  in  the  form  of  messages)  and  determining 
what,  if  anything,  should  be  done;  who  should  do  it;  and  how  long  it 
should  take.  The  Exec  then  coordinates  this  with  the  CAS  Director  and 
tracks  progress. 

Uhile  the  process  itself  is  simple,  acquiring  the  judgmental  skills 
required  to  make  the  proper  decisions  typically  takes  "experience"  and 
is  a  training  challenge.  Therefore,  a  training  mechanism  was  created 
which  could  provide  pseudoexperience  through  a  simulation  of  the  job  and 
its  environment,  including  typical  crisis  action  events.  It  is 
formatted  much  like  a  business  "in  basket"  exercise;  however,  it  can 
give  advice  on  handling  each  event,  provide  varied  means  to  complete  an 
action,  and  generate  feedback  on  that  action.  In  other  words,  the 
system  offers  guided  experience  with  feedback  which  a  typical  "in 
basket"  exercise  is  not  capable  of  providing. 

Selection  of  Hardware  and  Software 

Macintosh  was  the  only  hardware  considered  for  the  project  because  of 
its  prevalence  in  the  PACAF  Command  Center  and  its  proven  instructional 
capability.  SuperCard  was  chosen  as  the  primary  software  for  PACEXTRA 
because  it  was  efficient  to  use  (both  in  developing  graphics  and  other 
objects  and  in  programming),  inexpensive,  and  compatible  with  the 
Macintor'  hardware  already  on  hand  in  the  PACAF  CAS. 

Design 

The  design  process  for  PACEXTRA  was  three-phased,  consistent  with 
the  three  major  components  of  the  system.  First,  the  exercise 
system  was  designed,  then  the  authoring  system,  and  finally  the 
lessons.  All  design  was  constrained  by  the  chosen  hardware  and 
software. 
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Design  of  the  Exercise  System 

The  goal  of  the  Exercise  System  was  to  replicate,  as  realistically  as 
possible,  the  environment  in  which  the  CAS  Exec  performed  his/her  job  in 
order  to  provide  "real"  experience  in  the  events  he/she  would  encounter. 
The  desk  and  related  locations  and  items  in  the  CAS  "Ballroom"  were 
provided  graphically.  This  "ballroom"  contains  seven  desk  groupings, 
five  of  which  are  hexagonal  cluster  units.  The  other  two  are  a  set  of 
four  administrative  area  desks,  and  a  modified  U-shaped  desk  set  at 
which  the  directors  and  executive  officers  sit.  All  desk  groupings  are 
represented  on  the  PACEXTRA  screen. 

"Super  3D"  software  was  used  to  create  a  three-dimensional,  electronic, 
colored  model  of  the  complete  ballroom  complex,  including  furniture  and 
pertinent  equipment,  with  varied  level  of  detail.  Electronic 
"snapshots"  of  any  point  in  the  room  could  then  be  taken  from  any  eye 
point  and  at  any  range.  Realistic  perspective  views  from  the  Exec's 
desk  (including  phone,  books,  in-basket,  terminal,  etc.),  as  well  as  any 
location  in  the  room  that  is  pertinent  to  a  decision  being  made,  were 
thus  provided. 

To  present  and  control  these  views  and  the  variety  of  event-related 
items,  the  windows,  fields,  buttons,  and  graphic  objects  inherent  to 
SuperCard  were  used.  Thus,  exercise  characters,  their  words  in 
cartoon-like  bubbles,  messages,  phone  conversations,  book  pages,  etc. 
would  all  be  developed  as  overlaying  objects,  with  the  scene  itself  in 
the  SuperCard  "background. "  This  approach  would  be  efficient  to 
implement  and  would  facilitate  modifications  as  the  system  matured  and 
was  tested.  The  standard  640  X  480  pixel  Macintosh  screen  was  chosen  so 
the  exercises  could  be  run  on  any  Macintosh  II  hardware  with  sufficient 
memory  and  speed.  On-line  aids  were  provided  to  introduce  trainees  to 
PACEXTRA. 

Design  of  the  Authoring  System 

The  goal  of  the  PACEXTRA  authoring  system  is  to  allow  those  personnel 
supervising  the  training  for  the  CAS  Exec  job  to  create  and  modify 
exercises  with  only  an  hour  or  two  of  training  on  the  use  of 
PACEXTRA.  Since  the  heart  of  an  exercise  is  the  events  and  subsequent 
trainee  actions,  the  authoring  process  centers  on  a  graphic  "chain  of 
events."  The  chain  is  on  multiple  screen  "pages"  and  is  created  by 
linking  graphic  icons  representing  different  event  types  and  anticipated 
actions,  each  cne  labeled  with  its  identifier,  name,  and  source. 

"Double  clicking"  individual  icons  on  the  chain  will  take  the  author  to 
the  same  screen  the  trainee  will  eventually  see  when  that  event  occurs. 
The  author  can  then  add  all  the  details.  The  author  works  with  a 
graphic  chain  of  events  representing  the  sequencing  of  the  exercise  and 
providing  access  to  the  details  of  each  event  or  anticipated  action.  In 
addition,  it  is  possible  to  define  questions  the  trainee  may  ask  of 
event  characters  and  the  possible  responses  the  system  will  give. 


To  provide  a  realistic  set  of  people  within  the  CAS  with 

whom  the  Exec  interacts,  a  supporting  "cast  of  characters"  authoring 
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process  was  created.  The  author  combines  faces  and  bodies  (in 
Battle  Dress  Uniforms  (BDUs)  or  flight  suits)  and  assigns  each  to  an 
agency  and  desk  in  the  ballroom.  Thus,  a  broad  cross  section  of 
rank,  race,  and  sex  can  be  used  to  staff  the  CAS  with  a  different 
group  of  people  for  each  exercise.  This  approach  permits  an  author 
to  change  or  move  agencies,  thereby  remaining  current  with  the 
real  CAS  as  it  changes.  This  also  provides  a  more  interesting 
milieu  for  the  student  performing  the  CAS  Exec  duties.  An  on¬ 
line  authoring  aid  was  designed  so  exercise  authors  could  learn 
about  the  authoring  process  while  on  the  system.  This,  and  the  guide 
for  trainees,  is  available  in  print  separate  from  this  report. 

Design  of  the  Exercises 

Design  of  the  first  of  two  exercises  involved  meetings  with  PACAF 
DOC  and  DOX  staff  members.  In  these  meetings,  it  was  decided  that  the 
subject  of  the  first  exercise  would  be  an  insurrection  in  a  fictitious 
island-country  called  Neptune.  The  events  and  actions  would  involve 
establishing  a  Joint  Task  Force  (JTF)  and  supporting  the  JTF  with 
fighters,  tankers,  troops,  supplies,  equipment,  and  airlift.  The  second 
exercise  focuses  on  orienting  the  new  Exec  to  the  CAS  through  events 
involving  the  staff.  This  tests  the  application  of  this  system  to  a 
slightly  different  form  of  training  than  the  crisis  action  experience 
itself.  (In  practice,  the  trainee  ideally  goes  through  the  orientation 
exercise  first,  followed  by  the  Neptune  exercise.) 

The  exercise  design  process  begins  with  the  development  of  a  "short 
story"  on  paper.  (It  is  important  to  note  here  that  this  first  step  is 
not  a  storyboard,  but  rather  just  a  skeletal  narrative  of  the  events 
and  actions  planned.  A  storyboard  is  inappropriate  because  the 
subsequent  authoring  process  will  create  displays  which 
automatically  consist  of  chosen  ballroom  scenes,  people,  and  text. 

To  presketch  them  would  be  wasteful  and  counterproductive.  It 
could  be  said  that  the  on-system  authoring  process  automatically 
creates  a  storyboard  as  the  first  draft  of  the  exercise.)  Once  the 
story  is  developed,  the  cast  of  characters  is  created  on  the  system 
and  events  are  sequenced  on  the  on-screen  chain.  Finally,  the  details 
which  reflect  the  "short  story"  are  added.  Thus,  the  exercise  design 
and  development  process  become  one  when  the  story  is  drafted. 

Development 

The  PACEXTRA  development  process  consisted  of  five  major  steps: 

1.  Collecting  subject  matter  details; 

2.  Producing  the  required  graphics; 

3.  Developing  the  supporting  software; 

4.  Authoring  the  exercises;  and 

5.  Testing,  refining,  and  reporting  the  system. 

Each  step  is  addressed  in  the  following  paragraphs. 
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Collecting  Subject  Matter  Details 

Information  on  the  CAS  Exec's  job,  the  tools  he/she  employs,  and  the 
environment  in  which  he/she  works  was  obtained  through  visits  to 
PACAF/DOC.  Unclassified  documents  and  forms  were  collected; 
measurements  and  photos  of  the  ballroom  were  taken;  and 
discussions  were  conducted  with  staff  members,  particularly  those 
already  qualified  for  CAS  Exec  duty.  As  the  system  matured 
it  underwent  an  informal  formative  evaluation  by  these  people. 

Producing  the  Required  Graphics 

Photos  and  measurements  of  the  ballroom  and  its  equipment 
were  used  to  electronically  model  the  principal  CAS  physical 
components  to  scale  using  Super  3D  software.  This  included  the 
desks,  chairs,  printers,  and  large  screen  displays  throughout  most  of 
the  room  in  a  moderate  level  of  detail.  At  the  Exec's  desk,  detail 
was  taken  to  a  more  refined  level.  Including  the  individual  phone 
buttons  and  the  binder  rings  on  clipboards  and  notebooks.  Then 
electronic  "snapshots"  were  taken  of  Super  3D  ballroom  screens,  set 
up  to  represent  the  Exec's  line  of  sight  while  seated  at  the  desk, 
while  turned  to  talk  to  the  Director,  while  turned  to  talk  to  visitors, 
while  approaching  each  desk  cluster,  and  while  approaching  an 
individual  cluster  desk  position.  Capture  software  was  used  to  import 
the  electronic  snapshots  into  the  SuperCard  environment  as  card 
backgrounds. 

Super  3D  was  also  used  to  create  enlarged  views  of  the  Exec's  tools, 
such  as  the  telephone  receiver,  opened  and  closed  notebooks, 
message  clipboards,  computer  terminal,  etc.  These  were  captured 
and  imported  into  SuperCard  as  graphic  objects.  The  graphic 
functions  built  into  SuperCard  were  used  to  modify  and  embellish 
these  items  and  create  other  items  such  as  the  cartoon-like 
bubbles  which  enclose  the  words  spoken  by  scenario  characters. 

To  support  the  process  of  creating  a  cast  of  characters,  a  graphic 
artist  drew  monochrome  faces  and  bodies  using  SuperPaint.  These 
were  imported  to  SuperCard  where  they  were  sized,  colored,  and 
refined  to  create  a  library  of  24  faces  and  12  bodies. 

These  include  male,  female,  black,  white,  officer  (Major),  enlisted 
(MSgt.),  BDU,  and  flight  suit.  The  author  must  access  these  on 
pallets  while  using  the  "Casting  of  Characters"  feature. 

The  drawing  and  painting  features  of  SuperCard  (in  some  cases  aided 
by  Super  3D)  were  used  to  create  the  remaining  graphics.  These 
include  the  icons  and  background  for  the  event  chain;  large  "Help," 
"Stop,"  and  "Next/Past  Event"  buttons  for  the  trainee;  and  other  items 
on  behind-the-scenes  authoring  screens.  SuperCard's  buttons  and 
fields  were  used  extensively  in  this  area. 
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Developing  the  Supporting  Software 

As  explained  previously,  a  great  deal  of  the  PACEXTRA  development 
involved  using  the  nonprogramming  features  of  Supercard  to  create 
windows  and  cards,  draw  and  paint  graphics,  and  define  and  insert 
buttons  and  fields.  However,  these  hypermedia  functions  alone  are  only 
a  portion  of  PACEXTRA.  To  make  the  system  perform  as  it  does,  a 
significant  amount  of  software  code  (Supercard  script)  was  written. 
Approximately  1000  hours  of  programmer's  time  was  required  for  this. 

Due  to  the  object-oriented  nature  of  SuperCard,  the  unique 
PACEXTRA  code  (script)  is  contained  in  the  various  system  objects 
including  projects,  windows,  backgrounds,  cards,  fields,  buttons,  and 
graphic  objects.  The  PACEXTRA  Software  Document  (separate  from  this 
report)  describes  the  software  scheme  in  some  detail.  The  script 
itself  is  accessible  within  PACEXTRA  through  SuperCard  and  its 
subsystem,  SuperEdit. 

Authoring  the  Exercises 

The  two  exercises  were  developed  by  first  outlining  a  series  of 
events  on  paper  (similar  to  outling  a  short  story).  A  cast  of 
characters  was  then  developed  to  staff  the  CAS.  A  chain  of  events  and 
anticipated  actions  (responses  by  trainees)  was  developed  on  the  system, 
and  details  were  added  to  each  event  and  action.  These  details  included 
the  text  of  messages,  phone  calls,  and  direct  contact  with  CAS 
characters.  They  also  included  the  definition  of  any  conditions 
(trainee  actions)  which  are  to  influence  the  occurrence  of  specific 
events.  The  result  of  this  process,  and  the  reviews  and  refinements 
which  followed,  are  the  two  exercises  delivered  with  PACEXTRA. 

Testing,  Refining,  and  Reporting  the  System 

The  scope  of  this  project  did  not  allow  for  formal  testing  of 
PACEXTRA.  Testing  has  been  limited  to  that  done  by  Logicon  during 
development,  and  as  a  result  of  using  the  authoring  system  to 
develop  the  two  delivered  exercises.  Informal  formative  evaluations 
were  conducted  with  the  PACAF/DOC  staff. 

IV.  RESULTS 

This  project  resulted  in  the  production  of  an  experimental  prototype 
of  a  microcomputer-based  training  system  called  PACEXTRA.  It  runs 
on  a  Macintosh  II  computer  and  provides  individualized  exercise- 
based  training  to  those  preparing  to  perform  the  duties  of  Executive 
Officer  in  the  PACAF  CAS.  The  system  demonstrates  the  concept  of  using 
low-cost,  hypermedia  software  (SuperCard)  to  create  a  user-friendly 
exercise  authoring  tool,  the  implementation  of  which  results  in 
realistic  event-driven  exercises.  These  exercises  put  the  trainee  into 
a  highly  graphic  on-screen  environment,  which  is  expected  to  provide 
high  transfer  of  training  to  the  real  job  environment.  The  exercises 
challenge  the  trainee  with  events;  the  trainee  must  decide  what  action 
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should  be  taken.  The  system  provides  four  levels  of  advice  and  the 
means  of  taking  action  in  realistic  job-like  ways  such  as  sending 
messages  or  contacting  others,  either  directly  or  by  phone.  Feedback  on 
what  was  expected  and  whether  it  was  accomplished  are  provided  for  each 
event . 


This  project  focused  only  on  the  design  and  development  of  the  training 
system  and  included  no  evaluation  or  research  on  its  effectiveness.  Any 
such  future  research  by  AFHRL  or  others  will  be  reported  separately. 

V.  DISCUSSION  AND  CONCLUSIONS 

Lessons  Learned 

Informal  review  of  PACEXTRA  by  the  PACAF/DOC  personnel  has  been 
positive.  Indications  are  that  its  realistic  modeling  of  the  CAG 
Ballroom  and  the  events  which  occur  therein  will  provide 
effective  training  with  a  high  degree  of  transfer;  however,  this  has  yet 
to  be  proven. 

Use  of  the  authoring  system  by  the  Logicon  staff  has  shown  it  to  be 
easy  to  use  and  capable  of  creating  a  comprehensive  set  of  realistic 
events  and  anticipated  actions.  The  capability  to  easily  add,  move, 
modify,  and  delete  individual  events,  independent  of  other  events,  is 
significant.  It  makes  progressive  exercise  development  and 
revision  at  any  stage  of  development  very  practical  and  efficient. 

An  author  can  first  create  a  skeletal  exercise  (with  minimum 
events),  run  that  exercise  in  the  trainee  mode,  then  add  to  it  as  the 
trial  run  dictates.  Preliminary  versions  of  exercises  can  thus  be 
created  in  a  few  hours. 

SuperCard  has  proven  to  be  a  very  capable  software  utility, 
particularly  in  light  of  its  price  (under  $300).  However,  a  limitation 
is  the  time  it  takes  to  change  from  one  display  screen  to  another 
when  the  exercise  is  running.  This  is  particularly  true  if  the  display 
includes  multicolored  compound  graphics  (as  many  PACEXTRA 
screens  do).  PACEXTRA  was  tested  on  the  "basic,"  "ci,"  "cx,"  and  "fx" 
versions  of  the  Macintosh  II  microcomputer;  the  system  only  runs  at  a 
reasonable  speed  (average  two-second  screen  change)  on  the 
top-of-the-line  "fx"  model.  In  addition,  due  to  the  multiwindow 
approach  used  on  PACEXTRA,  the  hardware  must  include  a  full  eight 
megabytes  of  memory.  Otherwise,  the  system  will  freeze  up  when  enough 
windows  are  open  to  saturate  the  memory. 

Lessons  regarding  the  true  operational  utility  and  training 
effectiveness  of  PACEXTRA  must  await  its  application  in  the  field 
and  any  related  research. 

Advantages  and  Disadvantages  of  Approach 

The  advantage  of  employing  a  low-cost,  off-the-shelf,  object- 
oriented,  hypermedia-based  software  package  like  SuperCard  is  that 
it  provides  many  of  the  features  needed  for  interactive  training 
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systems,  yet  leaves  the  application  of  these  features  up  to 
the  imagination  and  ingenuity  of  the  training  system  designer/developer. 
The  ability  to  create,  position,  color,  re-size,  and  add  functionality 
to  windows,  cards,  buttons,  fields,  and  objects  without  programming,  and 
then  to  further  enhance  their  functions  with  scripts,  gives  the 
designer/developer  an  ideal  tool.  However,  the  scripting  language,  due 
to  its  interpretive  nature,  runs  very  slowly,  particularly  on  anything 
less  than  the  top-of-the-line  Macintosh  hardware  (fx  model). 

The  true  advantages  and  disadvantages  of  PACEXTRA  as  a  training  system 
will  not  be  known  until  it  is  tested  and  employed  in  the  field,  which  is 
beyond  the  scope  of  this  project.  However,  experience  in  applying  the 
PACEXTRA  exercise  auth''ring  system  has  shown  it  to  be  very  efficient  and 
easy  to  use. 

Potential  Application 

PACEXTRA  itself,  because  it  was  designed  specifically  to  train  only 
the  CAS  Exec  at  PACAF,  has  no  other  potential  direct  application. 
However,  the  approach  to  event-driven  exercise-based  training  which  it 
exemplifies  has  application  to  nearly  any  event-related  job.  For 
example,  it  could  be  used  to  train  store  clerks  to  handle  various 
customer  situations.  At  the  other  end  of  the  spectrum,  it  could  be  used 
to  give  space  crews  pseudoexperience  with  mission  events  which  require 
them  to  make  decisions  and  take  actions.  To  convert  PACEXTRA  to  train 
another  job  would  primarily  involve  creating  the  proper  graphic 
environment  and  the  on-the-job  tools.  The  underlying  PACEXTRA  system 
has  generic  application  potential. 

The  real  potential  of  the  type  of  training  system  PACEXTRA  represents  is 
to  teach  both  procedural-  or  rule-based  decision-making  and 
knowledge-based  decision-making,  particularly  because  it  provides  so 
much  of  the  complex  context  of  a  decision  environment.  This  is  in 
contrast  to  procedural  trainers  which  simply  list  "rules"  or  procedures 
with  little  sense  of  the  "knowledge"  that  is  gained  by  experience  in  the 
decision  context.  This  is  the  force  behind  expensive  "virtual  world" 
devices.  The  advantages  of  the  PACEXTRA  approach  is  its  low  cost, 
availability,  focus  on  coordination  of  activities  (i.e.,  teamwork),  and 
speed  of  exercise  authoring.  There  is  little  evidence  to  suggest  the 
true  need  for  the  "glamor"  of  high-tech  reality.  What  is  needed  is  the 
work-inspiring  nature  of  an  interactive  environment.  PACEXTRA  appears 
to  provide  this,  at  a  low  cost  and  with  minimal  effort  on  the  part  of 
scenario  authors.  Its  potential  seems  to  be  significant. 

Recommended  Follow-On 

As  described  elsewhere  in  this  report,  the  event-driven  exercise 
format  of  PACEXTRA  has  innumerable  other  potential  applications, 
and  the  authoring  system  which  it  incorporates  makes  it  relatively 
easy  for  trainers  to  create  and  modify  training  exercises.  A  natural 
follow-on,  therefore,  would  be  to  modify  the  system  (primarily  the 
authoring  system)  to  focus  on  another  job  and  its  environment  so 
exercises  could  be  created  to  train  that  other  job. 
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However,  an  even  more  useful  follow-on  would  be  to  experiment 
with  developing  a  higher-level  "authoring”  process  which  would 
permit  training  system  developers  to  create  their  own  exercise 
authoring  system  for  their  unique  jobs.  This  higher-level  authoring 
process  could  then  permit  the  exercise-based  training  exemplified 
in  PACEXTRA  to  be  applied  to  training  any  other  job,  without  going 
through  the  time  and  expense  of  building  the  supporting  tailored 
authoring  system  through  programming.  Rather,  the  trainers  could 
develop  their  tailored  authoring  system  themselves,  through  an 
interactive  process  similar  to  that  of  other  interactive  Macintosh 
applications.  The  result  could  add  another  dimension  to  the  media 
and  methods  available  in  technical  and  operational  training 
throughout  the  Air  Force  and  elsewhere.  (Note:  With  the  advent  of 
Windows  3.0  and  object-oriented  hypermedia  applications  such  as 
Toolbook,  this  approach  now  appears  practical  on  the  IBM  PC  and 
compatible  family  of  computers  as  well.) 
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